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1 Image processing device that codes multiple value images, characterized by the fact 
that it comprises a classification means that classifies the input image into two regions; i.e., the 
character region consisting of letters, figures, etc., and the halftone region of photographs, etc., 



' [Editor's note: Pagination in 



the right margin represents pagination in the original foreign text.] 



a block-making means that divides the input image classified into the character region as 

region as a result of the classification by the aforementioned classification means. 

2 image processing device that codes multiple value images, characters by the « 
,ha, it comprises a samp.ing means that draws representative values a, constant mtervals of the 

'Zproximation means ,ha, approximates ,„ curved surfaces the input image from the 

rKuito ::rp— 

inrage and the result of approximation by the aforementioned approximatton means 

and a coding means which codes directly when the result of computation by ft. 

PI aim 2 Iraeterized by the fact that ,. comprises a means to classify the coded data tnto 
decoded image using this additional informafion tn ease the aforementtoned addmonal 

approximation mL changes the dimens.ons of the curved —ft ; tg ft he d^rge 

in he concenhation of the image, and that i, reduces the curved surface when .-V 
changes and increases the curved surface in a monotone case where there are few changes. 

p P t™H gv planation of ttip, invention 

WB t^ll periainaftanimageprocesstngdevtceftateodesmuU^value 
manner. 
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PriM lagea are information tha, can assist in intuitively grasping the who.e. Com- 
ing images is widely popularized as an excellent me» to transm,, mfomtatton. However, 



because images generally contain an ex„emely large quantity of information, ,f .hey are dtrectly 
emitted to a emote locatiotr through communication network in the form of ongtna, rmage 
Z In or stored in vanous e,ee,ro„ic medra, the cos, and .he processing ..me become a 
Ittfcan. res.ric.ion. Therefore, a high efficiency coding aysrem to effic.en.ly compress, 
transmit or store image information is needed. 

Known htgh efficiency codmg systems include the predictive codmg system, the Mock 
coding system (ETC: Block Truncation Codmg), the direct conversion coding system, the vector 
quantization system, entropy coding system, etc. 

Problems to be solved by the invention 

Although various high efficrency coding systems have been proposed as menooned 
earlier, each haa i,s advantages and disadvantages depending on .he characters of e 
document image. None is good enough ,0 code images of documents wherem letters, hue 
drawings halftone images, etc., are mixed. t . 

The purpose of me present invention is to offer an tmage processing devrce that mamtams 
the high picture quality of the image information, can perfomr advanced compresston, and can 
code/decode in a short processing time. 

Means to solve the problems . 

Tbe present invention ia an image processing device that codes muttrple value ,m ges, 
and comprises a classification means (11 in Figure 1) that classifies the input image tnto two 

b o^aph , etc., a block-making means (12 in Figure 1) that drvrdes the input tmage Cashed 
1 Z character region aa a result of the classified by mis classified means ,m »*. 
consisting of N. X N2 (Nl and N2 are positive integer) elements, a firs, codmg mean (13 ,n 
Z« and Figure 3) ha, codes each of the blocks that are made into blocks by tbts block- 
X means, and a second coding means (14 in Figure 1 and Figure 4, .ha, codes he regton 
tha, has been classified into the halftone region as a result of the classtficatton by ,he 
aforementioned classification means. 

Also, i, is an tmage processing device characterized by the fact that ,« 
sampling means (6, in Figure 5, thai draws representative values a, constant tntervals f the 

put image, an approximation means (53 in Figure 5, that apptoximates m curved surface the 
Tp 1 e ftom the .suit of sampling, a computation means ,54 in Figure 5, .a, combes tbe 
Jproxim tion errors by comparing the input image and the result of ? p— ^ 
aforementioned approximation means, and a codmg means (55-58 m Ftgure 5), whtch codes 



9 

directly when the result of computation by this computation means is smaller than a prescribed 
threshold value, but codes by adding additional information in case it is larger. 

Also the device that decodes the results coded by the image processing device in 
accordance with the present invention comprises a means to classify (1 1 1 in Figure 1 1) the coded 
data into additional information and curved-surface approximate information, and a means to 
correct (1 14-117 in Figure 1 1) the decoded image using this additional information in case the 
aforementioned additional information is present. 

Also according to one mode of the present invention, the approximation means (53 in 
Figure 5) can have a constitution wherein it changes the dimensions of the curved surface 
corresponding to the change in the concentration of the image, so that it reduces the curved 
surface when there are a lot of changes and increases the curved surface in a monotone case 
where there are few changes. 



Function . 

The input image is classified into the character region and the halftone region by the 
region classification means, and the character region is divided into blocks by the block-making 
means The first coding means codes each block according to a coding system that is appropriate 
for the statistical characteristics of the character region. Also, the second coding means codes the 
halftone region according to a coding system that is appropriate for its statistical characteristics. 
In this manner, in the present invention the image is divided into regions and coded using the 
coding system that is appropriate to each of the regions. Therefore, advanced compression can be 
made while maintaining a high picture quality, and processing time can also be shortened. 

Also in the coding of the present invention, the sampling means takes samples and codes 
the picture element concentration at the sample point. Thus, the amount of coding can be 
compressed at an advanced level. Furthermore, when the difference (approximation difference) 
between the image the approximation means interpolated from the sample point by curved- 
surface approximation and the input image is larger than the prescribed threshold value, the 
coding means codes the approximation error as additional information and adds it to the coding 
information of the picture element concentration for the aforementioned sample point. Therefore, 
errors can be corrected by means of additional information at the time of decoding. Thus, 
decoding can be performed at a high quality. 

Also the present invention can vary the sampling depending on the charactenstics of the 
image- i.e., fine sampling for regions with severe concentration changes and rough sampling for 
monotone regions. In this manner, the batch size of curved-surface approximation can be made 
variable, and efficient coding can be realized. 



Application examples 

Figure 2 illustrates a schematic constitution of an image processing device wherein the 
present invention is applied to a digital copier. It comprises an image input part (21) such as an 
image scanner, an image processing part (22) that performs processing of images such as unage 
conversion or editing, an image output part (23) such as a printer, a coding/decoding means 
(CODEC) (24) that performs efficiency coding by compressing the image or decoding by the 
inverse processing, and a memory (25) that stores the coded images. 

Figure 1 illustrates the constitution of an application example that performs the coding of 
the present invention. It comprises a region classification means (1 1) that classifies the input 
image into the character region consisting of characters, figures, etc., and the halftone region 
such as photographs, a block-making means (12) that divides the input image of the reg.on 
classified into the character region as a result of the classification by this region classification 
means (1 1) into blocks of Nl X N2 (Nl and N2 are positive integers) picture elements, a 
character region coding means (13) that codes each of the blocks that are made into blocks by 
this block-making means (12), and a halftone region coding means (14) that codes the region that 
has been classified into the halftone region. Both the region classification means (11) and the 
block-making means (12) belong in the function of the image processing part (22) m Figure 2, 
and both the character region coding means (13) and the halftone region coding means (14) 
belong in the function of the coding/decoding means (24) in Figure 2. 

The document image input by the image input part (21) is separated by the region 
classification means (11) into the character region consisting of sentences, graphs, line drawings, 
etc and the halftone region including figures that expresses shades using halftone dots or 
dithers, photographs, etc. The separation may be automatically made using the characteristics of 
the image or may be indicated by the operator. 

The block-making means (301) divides the character region into blocks of Nl X N2 
picture elements. In the present application example, blocks of 4 X 4 picture elements were 

^^Each block is high efficiency coded by the character region coding means (1 3), and stored 

by the memory (25). 

Here we will explain the character region coding means (1 3) in detail. 

Figure 3 illustrates the character region coding means (13) in Figure 1 in detail. 

The character region coding means (1 3) comprises a block memory (302) that stores the 
blocks of character regions created by the block-making means (301) (12 in Figure 2) a mean 
concentration value computing means (303) that computes the mean concentration value within 
the block, a standard deviation computing means (304) that computes the standard deviation 
within the block, a label determination means (305) that examines the concentration changes of 



the image and determines the charae.eristics of .he image based on me compute* mean 
!o „ 1 ation value and the standard deviation vahte, a one gradation block approx.m hon 
ml (one gradation BTC) (306), which is nsed to eode the b,oc k when the block 
M S in a pi flahet ® [bactrground par,]) where the — don change .s very mtld as a 
e of .he eJmination by .he .abe, detention means (305), a two 
Elation coding means (2 gradation BTC) (307), which is used to code the block when he 
7<2 Lgs in (label • [quasi background par.]) where the concen.ra.ion change ,s re a vely 

l p « labe ® [cha.ac.er par,,, where .he eoncen.ra.ion change is severe, a Hnf^an eodmg 
La s (309) mat Impression codes .he gradation information from the one gradafon BTC 
rl radation BTC (307) or Muear quantization part (308), anMMR codmgmeans (3.0) 

(307) and linear quan.iaa.ion par. (308) together, and a code assembly par. (311) .ha. assembles 
the codes. 

Next we will explain the action of (he character region codtng means. 

haractcr re^on Mock s.ored in ,he block memory (302) has .he mean concentration 
value Z Z means (303) compu.e the mean concentration value (Ave), and has the standard 

mcw (305) will determine the label as follows based on the mean —.,0 alu (Ave), 
standard deviation (Sd) and appropriately dc.em.ined .hrcshold values (Th.) and (Th2). 
Label ® (Background part) : Ave > Thl 
Label © (Quasi background part) : Ave > Thl, and 

Sd<Th2 

Label ® (Character part) : Ave > Thl , and 

Sd > Th2 

When a block is determined .0 be label ® by the label de,emtination means (305), the 
Mock is a flat section where the concentration change is mild, Based on .Ms statical 

1 eristic the inside of .he block is a PP roxima.ed in one gradation. Therefore, usmg th one 
"block approximation coding means (306), the mean concentration value wdhrn the 
block is assumed to be the gradation of the block. 

When a block is determined to be label » by the label de.ermma.ron means (30 , .be 

chamc.eris.ie, the insrde ofthe block is approximated in two gradations. There o« g* 

M A\ na mpatr; G(m the gradation information (Xo) ana iaj 
aradation block approximation coding means / h & Aa 

",) whfch indicate! which representative value approximates each picture element, , obtamed. 



When a block is determined to be label ® by the label determination means (305), the 
blook is a par, where me concentration change is severe. As a process appropnate for thts 
statistical characteristic, linear qnantization is adopted. That is, the linear quantum part 08) 
performs linear quantization using 16 gradations of qnantization .evels obtamod by se..,ng he 
levels by setting one level for the background part and equally dividing the other part tmo 15 
levels 

' The Huffman coding means (309) performs Huffman coding, which makes compress.on 
by means of variable length code assignment of the gradation information output by the one 
gradation block approximation coding means (306), two gradation block approx.maUon codmg 
means (307) and linear quantization part (308). 

Also, the MMR coding means (310) asstgns 0 to a.l of the picture elements of the outpu 
from the labe. <D one gradation block approximation coding means (306), an d assigns 1 to aU of 
the picture elements of the output from the label • linear quantization part (308) based on the 
resolving power information obtained by the label ® two gradation block approxtmatton codmg 

means (307), and MMR-codes all these. „,m„„„ 
The code assembly part (311) generates code output based on the output of the Huffman 
coding means (309) and the MMR coding means (310). Figure 4 illustrates the code structure 
Tlbled to store one character region in the memory (25). 1. codes the following tnformatto. 
Header infotmation H: Image size in the character region and location of the regton 
Label ®: Gradation information G 

Label ©: Gradation information G and resolving power information P 

Label ®: Gradation information G 

The character image coded in this manner is stored in the memory (25). 

Next the decoding of information on the character region that is coded and stored ts 
performed by reproducing the concentration distribution of each block from the t gradahon 
nfotmation and resolvmg power information. Each of dte blocks of label «> and label ® 
reproduced in one gradation and two gradation (representative values). 
,6 representative values reproduce the concentration value of each ptcture element m the block. 
Next we will explain the processing of the halftone region. 

Figme 5 illustrates an example of the constitution of the halftone region codmg means 
(,4) in Figure 1. It comprises a sampling part (51), predictive coding means (52), curved surface 
approximation means (53), approximate etror computation part (54), nonlinear quannza Uo 
part (55), Huffman coding means (56), mode determination means (57) and code assembly part 

t58> ' According to the algorithm of the coding in this example, first, the sampling part (51) 
takes samples in a grid form to use the correlation between concentration values tn verttoal, 



horizon*, and diagonal direction. As «tiis sampling position is fixed, no information .ha, 

approxin^d hy .he carved surface approxinra.ion means (53) .ha. uses congn,e„, teri.ary 

olession). The mean square error he,ween .he original image ,„ the reg.on and .he 
I"Jimage is .hen oh.a,„ed for each region hy .he approximarion error comparing 

"""" The mode determination means (57), if .he mean square error is larger man .he .hreshold 
means (56). 

Next each process will be explained in more detail. 

approximation is made with 19 representative values. Next, a Huffman 
reg ,on (.he d,amo„d-shaped region wi.h ,he four vertices as .he verhces, 



The mode determination means (57) determines that the mode ts I (tegton tha needs 
additional information) if the mean square error (MSB) is iarger than .he threshold vata (TH) 
Z Z *e mode is 2 (region ,ha« needs no addi.io.ra. infomration) if * is .ess ,h™ he threshold 
1! TH Here, the mode information .ha. prepares the mode information (MD)tha, tndrcates 
wth mode the Ipproximated region belongs in is binary image of 0 and 1, so MMR ecdmg rs 
adopted at the time of coding. 

When .he determination by the mode determination means (57) . a mode 1 the 
approximation error is coded by means of .he nonhnear quantization par, (55, an tire Huffman 
coding means (56). In .his application examp.e, .he nonhnear quanttzatton part (5 5) 
approximates with . 1 representative va.ues. Then, .he Hulmtan codmg means (56) codes by 
assigning a Huffman code to each representative value. 

8 As illustrated in Figure ,0 me code assemb.y part (58) assemb.es rn code form the 
predicted error (S) of tire sample point, mode information (M), and the appro— error, « 

„ tpu, The header information (H) is inserted at the begtnmng of the coded output. The header 
Nation ,s information such as the position and size of .he region when .he tmage ,n the 
halftone region is coded. The additional information is only needed ,„ case of mode 1. 

As explamed above, the decoding of the .mage of the code, halftone regton o n be 
executed by a decoding means, which has, for instance, a constitution such as lustr* d ,n 
Fisurell From the code information of, he code stmctme illustrated tn Frgure 10, the 
I! on separating means (1 1 1) separates the prediction error information , (S>, ad tinmo, 
to formation (F), mode information (M), e,o, and decodes ,hese in respective decodtng m«m 
2H 14) ami (1.7). Based on ,he concentration information of me sample pom. output from 
de^ ilg meals ( ,2) of ,he predictive coding code, tire curved surface appr„x,ma,,o» means 
~,a,es by means of congruen, .ertiary batch, and obtains .he concentratton value of 
11 o icmTe elemen When the mode information is mode 2, tire selection means (1 16, w,„ 

decoder, output and mate selection output. When the mode info—a mode ,U* 
Section means (1 15) wiU correct .he ou. P u, of the curved surface approx— means (1.3) 
Xme approximation error vahre, which has been decoded by tire addition, tnformahon 
Ilding means (1 .4), and the selection means (1.6) wil, select and output the output of the 
correction means (116) (sic; (115)] as the decoded output. 

,„ the aforementioned application example (the first apphoatton , example), we xplaneda 
case where tire batch size of the ha.fto„e region is fixed, that is, when the sample pom* «fad 
Z a certain space. However, ,. ts also posstWe ,„ make the sample pom. vanable Md change 
the batch size. 
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Next, as the second application example, let us explain the system of variable batch size 

(and therefore sample points). 

In this system, the image to be coded is divided into many diamonds (parallelogram- 
shaped), that is, batches. The points that are needed ,0 constitute each batch am extracted as the 
sample points. The concentration information of these sample points is stored. In decodtng, the 
concentration value within .be batch is kept based on the concentration infon.at.on of tints 
sample point. By executing this process for all the batches, the image is decoded. Mm 
the batch size is based on the fad that a halftone image comprises both parts the. have severe 
concentration changes such as the aurface of an object and regions with monotone concentrate 
distribution such as the background. Efficient coding can be realized if fine samphng > apphcd 
to the regions with severe concentration changes and rough sampling is appl.ed to monotone 

reg ' 0nS The point where the second application example atrncturally (functionally) differs from 
the firs, application example illustrated in Figure 5 is mainly that selecting the sample potnts m 
,he sampling means (51) is based on the batch size. The other parts of constitution are 

approximately the same. 

The batch is a diamond-shaped region with four sample points aa verttces such as 
illustrated in Fignre 12. It is a square inclined by 45" .0 .he arrangement of picture elements. The 
sample points are picked a. the center of four picture elements in virtually the same manne, -u 
the first application example. The mean value of these four picture elements ts assumed to be the 

concentration value of the sample point. 

The minimum batch size is m = 2. Corresponding to the changes of concentration at. 
illustrated in Figure 12, in a state where .he point that is located a. the top of the batch ,s xed 
the batch is expanded in a shape of hanging bell, and n is dc.em.ined wrth.n .he range of 0 to 7. 

The procedure for the other determination is as follows: 
0 As illustrated in Figure 13, tire concentration ^.information in the state of mintmum ba«ch 
size (n = 0) is obtained as reference information. 

,,) The batch with n - 1 is divided into minimum batch size, and the concentration s ope ,s 

obtamed in the sante manner as reference infomtation for each of me minimum batches 

ni) If the absolute value of tire difference between tite absolute value :of c— „ slope -4 

th absolute value of reference information is larger than .he threshold value (TH1), n - 0 wtll 

assumed ,o be .he batch size. If it is a, or less than the threshold value, the 

nrentioned above will be executed with n - 2. However, if the mtmmum batch where the batch 

already hi. is included, it is assumed that n - 0 frrespective of the concentration slope. 

IV) Varying n from 0 to 7, the batch size is determined. 
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Once the batch size is determined, the sample point ean be extracted. Predictive coding » 
done for each sample point. Predictive coding can be done in the same manner as explained for 
the first application example. However, unlike the first application example, sampling is done 
while changing the batch size. Thus, there are two issues. That is, <D four points to be used for 
prediction do not always exist, and ® there may be cases where no points exist. 

Considering these issues, we will discuss below how to deal with each of the issues. 
With regard to the problem CD, the mean value is obtained by using only the existing point. 
With regard to the problem ©, the value of the vertex at the top of the batch is used as the 
predicted value. In case of the top level on the screen, no value exists even for the vertex ocated 
at the top. Thus, the median of usable concentration value width is assumed to be the predicted 

ValU6 ' In the coding structure, information on the coordinates of the top vertex of the batch and 
the batch size will further be added to the structure in Figure 10. 

Effect of the invention . . 

According to the present invention, the document image as the object is classified into 
two regions of a halftone region and a character region, and the coding system that is appropriate 
to the statistical characteristics of each region is used. Therefore, advanced compression while 
maintaining high picture quality can be realized. 

Also, according to the present invention, as it is constituted that in the coding of the 
halftone region, the region surrounded by four sampled points is approximated by apptying 

ngruenttertiary batch, andwhen the approximation error is larger than a fixed threshold value, 
3r —on of the approximated image is added, the improvement of picture quality can 

^ ^Furthermore, in the mode of the present invention where, in the coding of the halftone 
region, fine sampling is done for the aforementioned sampling points in a region with severe 
concentration changes of the image and rough sampling is done in a monotone region, more 
efficient coding will be realized. 

Briefjejcription of the figures 

Figure 1 is a block diagram that illustrates the constitution of the coding means of the 
image processing device in an application example of the present invention. 

Figure 2 is a diagram that illustrates the schematic constitution of an unage processing 
device wherein the present invention is applied to a digital copier. 

Figure 3 is a diagram that illustrates an example of constitution of the character region 

coding means in Figure 1 . 



• w 

Figure 4 is a diagram that illustrates the code constitution of the eodes that the character 

reeion coding means outputs. 

Figure 5 is a diagrcm that illustrates au example of constitution of the halftone reg,on 

coding means in Figure 1 . . 

Figure 6 is a diagram that illustrates the sampling points of the fixed density samphng. 
Figure 7 is a diagram that illustrates the reference sample points for predictive coding. 
Figure 8 is a diagram that illustrates the coordinate system of the congruent tertiary batch. 
Figure 9 is a diagram to explain the mean square errors between original image and 

approximated image within a region. 

Figure 10 is a diagram that illustrates the code structure in the halftone region. 
Figure 1 1 is a diagram that illustrates the constitution to decode the halftone region. 
Figure 12 is a diagram to explain how to determine the batch size. 
Figure 13 is a diagram to explain the concentration slope. 
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